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ABSTRACT 



Following the proposal of the Chancellor of the California 
Community to shift from attendance -based funding to performance -based 
funding, Palomar College (California) has articulated its intention to judge 
its c[uality and formulate its policies primarily on learning outcomes. Out of 
this imperative, the Assessment of Learning Project (ALP) was born. The 
overall goals are: (1) to establish ongoing, interdepartmental learning 

assessments, (2) to improve existing assessment methods, specific to various 
disciplines and programs, and (3) to modify the current curriculum, based on 
what was learned in the assessment process. ALP surveyed Palomar faculty, and 
conducted detailed interviews with a subset of faculty, to determine 
attitudes toward current learning assessment practices. Survey findings 
showed that 74% of the faculty agreed with the statement that the college 
should develop a method to assess skills and knowledge, but 53% of the 
respondents felt they did not have enough resources to adecpaately assess 
their students. Interviews revealed that about half the faculty use a formal 
assessment process at the beginning of their courses, most faculty use a 
variety of assessment instruments in addition to tests, and about one-third 
of those interviewed compare the progress of their current class with that of 
previous classes. The report includes five recommendations made as a result 
of the faculty input. The survey instrument, interview questions, and 
responses are appended. (VF) 
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Introduction 

There has been much discussion in the past year of "assessment" and "learning outcomes." 
The Chancellor of the California Community Colleges has proposed that colleges be 
funded in part on the basis of learning outcomes. Similar ideas have been discussed for 
many years at Palomar. Palomar’s Mission Statement expresses the goal this way: "We 
evaluate the relevant skills and knowledge of all of our students so as to guide them 
toward meaningful and productive educational experiences, patterned to develop their 
abilites as effectively as their preparation allows. We will evaluate our own 
performance in terms of our contribution to student learning and success." Our 
Vision Statement, adopted in 1991, projects a future in which "Palomar College judges its 
work and its programs and formulates its policies primarily on the basis of learning 
outcomes and has a comprehensive program for assessing those outcomes and 
responding to its findings." The Vision Statement encourages the state of California to 
"reduce reliance on attendance-based funding and reward performance as measured by 
learning outcomes." 
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Proposals to reward learning outcomes have raised important questions. Many faculty fear 
the prospect of "outcomes" defined by bureaucratic agencies for purposes of political 
"accountability" that fail to serve the interests of students. A highly standardized 
assessment regime in which faculty are encouraged to "teach to the test" would not 
promote student learning or long-term success. These fears could be well founded if we 
wait passively for outside agencies to impose their goals upon us. If that happens, it will 
be very difficult to reverse the momentum of policies already underway. The goal of 
learning assessment as it has been defined at Palomar has been to empower faculty, staff, 
and students to promote student learning. As Dr. George Boggs has written recently, 
"educators rather than legislators should establish learning outcome goals. This is a 
challenge that educational leaders must accept." 

It was to accept the challenge of developing a faculty-generated definition of learning 
outcomes and the means of assessment that could measure those outcomes that we applied 
last year to the Educational Master Planning Committee for an innovation grant to 
establish the Assessment of Learning Project (ALP). Our first task was to survey our own 
faculty both on their assessment practices and their views on the need for developing 
further assessment tools. 

Purpose (Tod ofPase) 

The immediate purpose of our project for the fall semester was to survey Palomar’s 
faculty in order to discover what they really think about assessment of learning as we 
conduct it now and as we should in the future and to formulate recommendations for 
further action based on what we found; To this end, we distributed written surveys to all 
faculty and conducted detailed personal interviews with a random group of full-time 
faculty. The results of these surveys and interviews are summarized below and given in 
more detail in the appendices. We were informed in this work by what we have learned of 
best practices in assessment at other colleges. 

Our larger goal, which obviously will not be accomplished this year, can be summarized 
in three parts; (1) to establish college-level assessment of learning outcomes common to 
many departments or programs; (2) to raise the quality of ongoing assessment within 
existing departments and programs, and (3) to improve student learning through 
developing, applying, and modifying curriculum and instructional methods based on what 
is learned in the assessment process. An integrated approach will allow the college to 
develop classroom-level assessment that contributes to the achievement of core learning 
along with college-wide assessment that provides valid evidence of the value added by 
courses and programs. 

We plan to submit a separate report on best practices in assessment at other colleges. 
Recommendations (Top of Page) 

We recommend that Palomar College move without delay to do the following: 

1. Establish college-wide assessment of common skills on a continuing basis. 



• Goals: 



• Refine the list of common skills defined in this report. 

• Develop faculty-based assessment processes and standards for 
consistent and credible assessment of these skills across the 
curriculum before our next accreditation self-study in the academic 
year 2001-2. 

• Establish college-wide assessment of these common skills by 2005, 
the target date of our Vision Statement. 

• Activities: 

• Survey best practices in assessment at other colleges 

• In the fall semester of 1998, conduct focus groups with faculty, 
administrators, staff, and students. 

• In the spring semester of 1999, develop pilot assessments and 
standards for their evaluation. 

• In the fall semester of 1999, conduct and evaluate pilot 
assessments. 

• In the spring semester of 2000, evaluate the results and plan future 
activities. 

Most of Palomar’s faculty agree on a body of skills that most or all of our students should 
master before graduating. And of those faculty who responded to the assessment survey, 
74 percent agreed or strongly agreed with the statement that Palomar should develop a 
means to assess the knowledge and skills of all AA candidates and transfer students. At 
the conclusion of these recommendations we present a list of common skills that we 
believe can be further refined and defined. These common definitions will allow us to 
proceed to develop tools by which our faculty can gather more reliable and usable 
information about the abilities and potentials of our students than we can today. We have 
set our next accreditation institutional self-study as a target date for developing an 
assessment plan because institutional outcomes assessment is now part of the institutional 
effectiveness standard that will be used for our next accreditation. 

2. Create standards for assessment in the classroom of skills particular to 
specific disciplines, certificates or vocational training. 

• Goals: 

• Establish standards for evaluating assessment methods. 

• Establish a process for ongoing experimentation and feedback on 
assessment. 
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• Activities: 



‘ Survey best practices in assessment at other colleges. 

‘ In the fall semester of 1998, conduct focus groups with faculty, 
administrators, staff, and students. 

• In the spring semester of 1999, develop pilot assessments and 
standards for their evaluation. 

• In the fall semester of 1999, conduct and evaluate pilot 
assessments. 

• In the spring semester of 2000, evaluate the results and plan future 
activities. 

While many common skills can be reasonably assessed at the college level, many of our 
courses and programs need to assess skills in a distinctive context of a particular discipline 
or task. All of the faculty we heard from devote a great deal of time and energy to trying 
to access student learning fairly and accurately. However, most faculty expressed some 
frustration that they do not always have either the tools or the standards to do the job as 
effectively as they might. In the survey, 53 percent of faculty said that they do not have 
enough resources to adequately assess their students. The results of classroom assessment 
are not readily available to other faculty. We believe that assessment across the board 
would be improved by ongoing communication among faculty about the means, 
standards, and tools of assessment. We believe that the best standards to guide our faculty 
in developing more creative and effective assessment will be standards developed by and 
for our faculty. 

5. Use assessment of student learning to evaluate and improve the 
curriculum, 

• Goals: 

• Integrate assessment into the curriculum development and 
evaluation process. 

• Modify the curriculum to enhance student learning. 

• Activities: 

' Survey best practices in integrating assessment into curriculum 
development. 

' Conduct small-scale tests of integrating assessment into curriculum 
in the classroom and other learning environments. 

Both the survey and the interview results strongly suggest that teachers at Palomar already 

ERIC 
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use assessment of student learning in order to develop and modify their courses and their 
teaching. But many faculty express frustration that they have little information and less 
control over the students who enter their classes or the experiences that build on their class 
work. The vast majority of faculty members we interviewed proposed ideas for gathering 
better information to be used in guidance and placement so that students would be fully 
prepared for the courses they attempt. If we can design into our curriculum paths that our 
students can more realistically travel, we can raise the level of student success without 
sacrificing excellence. 

4. Involve faculty and staff in ongoing discussions and decision-making with 
respect to assessment issues and inform them about best practices and 
current research. 

• Goals: 

• Inform faculty and staff concerning the best evidence with respect 
to learning assessment. 

• Generate widespread involvement among faculty across 
departments and divisions in formulating assessment plans and 
priorities. 

• Activities: 

• Conduct or sponsor professional development activities to inform 
faculty of current issues and research in assessment. 

• Maintain content of an assessment site on Palomar's Internet 
homepage. 

• Establish interdisciplinary teams of faculty and staff to develop 
plans for assessment. 

Several members of our faculty expressed their concern in interviews about the prospect 
of externally mandated assessment that would add paper work and reporting requirements 
without increasing our ability to promote student learning and success. These concerns 
echo the testimony of experts who have studied assessment programs at various campuses. 
The best evidence indicates that assessment programs adopted merely to respond to 
external mandates fail to serve student interests. On the other hand, assessment that is 
developed by the faculty at a college can change the learning environment for the better 
and empower faculty with new tools to promote learning. An effective assessment 
program for Palomar College must be designed and generated by the faculty and staff of 
Palomar College. And the process of developing such a program must involve an ongoing 
conversation among the faculty about the priorities and choices that shape it. 

5. Allocate sufficient funds to support the foregoing goals and activities. 

• Goals: All of the above. 
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• Activities: 



• Allocate at least $90,000.00 for the two academic years 1998-99 
and 1999-2000 to support the above assessment activities. 

• Fund released time for an Assessment of Learning Project Team, to 
consist of at least 50% released time for one faculty member and 
25% released time for three others, representing different divisions. 

• Aggressively seek outside funding to expand and hasten assessment 
development. 

• Designate an academic dean and the Director of Matriculation to 
facilitate the work of the Assessment of Learning Project. 

Developing an effective assessment program that can change the learning environment for 
the better will not require an enormous investment of resources. But it will require a 
constant, moderate investment of resources. In the long run, assessment is a good 
investment, one that will reap returns both in terms of Teaming and of revenue to the 
college. But the benefits of a locally designed and controlled assessment program can be 
achieved only by making a commitment to develop such a program. Palomar has been 
committed for year, on paper, to precisely such a task. It is time to do what we say we 
should. 



List of Common Skills (Toy of Pase) 

• Communication 
' Speaking 

' Listening 
' Reading 

• Writing 

• Cognition 

• Creative Thinking 

' Analysis and Synthesis 

• Problem Solving 

' Quantitative Reasoning 




• Interpersonal Behavior 
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' Social Interaction 



• Teamwork 

• Effective Citizenship 
' Intrapersonal Behavior 

• Transfer of Knowledge and Skills to a New Context 

• Aesthetic Responsiveness 



Summary of Faculty Survey (Top ofPaee) 

The Assessment of Learning Project (ALP) surveyed the faculty of Palomar College on 
assessment techniques that they are presently using and their ideal assessment program for 
the future. A total of 1,219 surveys were sent out to all full-time and adjunct faculty. Two 
hundred fifty-five of the surveys were completed and returned. Tables displaying the 
results of the survey are included in appendix B. 

Varied assessment techniques are employed by the faculty to assess a student's knowledge 
of the subject matter. Most of the techniques are traditional pen and paper tests such as 
multiple choice, fill in, calculation tests, essays and other writing assignments. These are 
used for both grades and student feedback in many cases. Non-traditional assessment 
techniques including portfolios, performances, demonstrations and classroom assessment 
techniques are used by many of the faculty. 

The list of skills appearing in the table on the following page was given to the faculty 
members and they were asked whether the skill was an intended learning outcome, and 
how often it was directly assessed. All of the skills except three-mathematics, computer 
literacy and aesthetic responsiveness-are intended learning outcomes for more than fifty 
percent of the faculty who responded. These skills embrace cognition, communication, 
inter-personal skills, intra-personal skills, and aesthetic skills. 

Nearly forty-three percent of the faculty surveyed either strongly agreed or agreed with 
the statement that assessment techniques are used to find out if current students in a given 
course are learning more than students in previous semesters. Nearly thirty-one percent 
disagreed or strongly disagreed. The remainder had no opinion. 

When asked if Palomar College adequately assesses student skills and knowledge before 
they complete their studies here, nearly twenty-eight percent of the faculty agreed or 
strongly agreed whereas, twenty-nine percent disagreed or strongly disagreed and 
forty-three percent had no opinion. Almost fifty percent of the faculty surveyed agree that 
Palomar College ensures that our AA graduates possess a minimum level of skills and 
knowledge appropriate to an AA degree. Seventy-four percent of the faculty surveyed 
strongly agreed or agreed with the statement that Palomar College should develop a 



means to assess knowledge and skills of all AA candidates and our students 
transferring to four-year institutions. 

Finally, fifty-three percent of the faculty report that they do not have enough resources to 
adequately assess their students. 



Skill 


Number of Respondents 


Percent Who Indicate the 
Skill Is an Intended 
Outcome 


Reading 


230 I 

i 


70.4 i 


Writing 


238 1 


80.7 


Mathematics 


219 


36.5 1 


Listening 


230 


81.7 1 


Speaking 


227 


63.9 


, Creative Thinking 


232 


81.0 


Problem Solving 


241 


84.2 


Analysis and Synthesis 


231 


88.3 

) 


1 Reasoning 


235 


91.9 


I Aesthetic Responsiveness 


210 


37.1 


i Computer Literacy 


224 i 


42.9 


1 Social Interaction and/or Teamwork 


233 


77.7 1 


! Value Judgements and Independent 
j Decision Making 


226 


77.0 


^ Transfer of Knowledge and Skills to a 
new Context 


232 


88.8 


; Effective Citizenship 


216 1 


56.0 


' Other (please specify) 


48 


77.1 j 



Summary of Faculty Interviews (Tod of Page) 

The Assessment of Learning Project Team also interviewed twenty-nine faculty members, 
covering a wide range of departments, as a follow-up to the survey. The results of these 
interviews are attached as an appendix, but a summary is presented here: 

• Overall, about half of the faculty interviewed are using a formal entry assessment 
at the beginning of their courses. In addition, some faculty informally assess students 
at the beginning of their classes. 
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• A variety of assessment techniques are used by the faculty interviewed. Most use 
traditional testing, while other techniques include essays, projects, book reports, and 
portfolios. These are listed in the appendix. 

• Other than assigning grades, most of the faculty interviewed use assessments of 
student learning to help students in different ways. For example, 1/4 of faculty use 
results to give feedback to students, and another 1/4 use the results to make 
improvements in their teaching and in their course curriculum. Nearly half use the 
results for other creative purposes. See the appendix for different uses. There was 
only one faculty member who said that s/he does not use the results for anything 
other than grading. 

• About 1/3 of the interviewees have a formal way of finding out whether their 
students this semester are learning either more or less than their students from 
previous semesters. Slightly less than 1/3 have an informal way of ascertaining this, 
and about 1/3 responded that they don't have a way of comparing this semester's 
results with previous semesters. 

• The majority of those interviewed are assessing the following skills throughout the 
semester: 

Reading 

Writing 

Creative Thinking 
Listening 
Speaking 
Reasoning 



Problem Solving 
Analysis and Synthesis 

Transfer of Knowledge and Skills to a New 
Context 

Computer Literacy 

Effective Citizenship 

Social Interaction and/or Teamwork 

Value Judgments and Independent Decision Making 
About half are not assessing Mathematics or Aesthetic Responsiveness. 



• The majority of faculty interviewed said that by using computerized assessment 
tests, student interviews, and personal guidance, Palomar can assess the knowledge 
and skills of incoming students in order to place them in the proper programs and 
provide guidance toward a successful college career. 

• As to how best to administer the assessment of the above noted skills, the majority 
of those interviewed suggested that assessment for reading, writing, and mathematics 
should be done at the college-wide level, and the rest should be done in the 
classroom. Only a very few of those interviewed believe that it should be done at the 
department or division level. The majority of faculty who responded to this question 
said that these skills should be assessed continuously, and about an equal number of 
faculty suggested pre- and post-testing which could include methods such as a 
written instrument, interview, and computerized testing. 

• Some faculty said that by tracking students' and instructors' records we should be 
able to find out whether our students this semester are learning more or less than 
students in previous semesters. But at the same time, faculty believe that it is very 
difficult to do and that grades are not a good indicator. 

• Some faculty suggested that departments need to do self-assessments to evaluate 
the performance of their individual programs. Other suggestions include creating a 
new assessment vehicle, comparing pre- and post-test results with other colleges, and 
tracking students. 

• See the appendix for a detailed listing of skills that faculty from each department 
believe the educated layperson should acquire from their respective disciplines. 

There were skills cited that were common to several disciplines, even among those 
disciplines which aren't generally skills-oriented. These include communication, 
reading, writing, listening, reasoning, and problem solving. 
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Appendix A - Survey Instrument 
Appendix B - Survey Responses 
(Tod ofPase) 



Faculty Assessment Survey Results | 




Fall 1997 I 


Part B 





How frequently do you use the following assessment techniques in your classes each semester? Are they used primarily for determining 
grades, providing feedback, or both? 



Percentage Distribution 



jAssessment Technique 


1 Numberij 


0 1 





4-7 II 8-1 2i| 


13+1 


|1 . Multiple Choice tests 


1 ! 


26.1 1 


26.6 i| 


34.9 II 


3.3 1 














[2. Fill-in tests 


;| 223 i| 


47.1 :| 


25.6 ;| 


16.1 || 


4.5 1 


1 .... 

I 


||3. Calculation Tests 


|| 220 i| 


63.6 ;| 


15.5^1 


14.1 II 4.1 || 


2.7 i 



; 4. Essay Tests 



225 35?!;^ 35?T^ 204:^ 4.9 l| 4.4 1 



5. Essays or other extended; 


221 1 


23.1 


41.2 ! 


18.1 ; 


7.2 1 


10.4 


writing assignments J 















||6. Formal presentations 



214 i| 40.2 : 47T1 



8.4 il f:4 



|7. Performances 


1 207 ; 


1 67.1 ; 


1 11-1 i 


1 10 - 6 : 


1 5.8 


1 6.3] 




|8. Demonstrations j 


213 


i 


I 25.4 i 


10.3 


I 6.1 i 


1 6.6 1 


1 

i 

I 


||9. Portfolio assignments ; 


1 209 


1 71.8; 


I... 


1 


1 3.3 


1 2.9) 


i 

! 














|10. Journal assignments 


1 208 


1 57.2 


I 16.8 i 


1 7-7 


1 6.3 ; 


1 




1 1 1 . Classroom assessment 


214 1 


36.4; 


22.4 i 


224 


7.01 


11.7 


: techniques 














! 

1 


n2. Research papers | 


1 213 


1 45.5 -| 


I 36.2 i 


1 12.2 


i 6.2 1 


1 6.9 1 


j 


>|1 3. Critical analyses 1 


206 i 


1 38.8 :| 


I 11.2 


1 10.2 


1 19-4 1 


1 ^20.4] 
















|14. Other 


75 


1 64.0 :| 


1 ^ 


1 4.0 


1 67! 


1 22 T j 



Percentage Distribution 
(for those with 1 or more use) ji 



Numberij Grades|| Feedback:| Both; 



1 ;; 170] 


L^il 


1 2.9 II 


r 58.2) 










1 iidi 


1 63.6 il 




1 60.9 1 




1 75) 


1 28.0 II 


! 5-3 1! 


66.71 




1 136 II 29.4 1 


1 .2:9.1 


1 67.6 j 










157 i 

1 


23.6 1 

i 

1 


4.5 1 


72.0 1 


1 ! 


1 29.8 { 


1 17.411 


L “.9J 




1 65 1 


12.3 il 


1 23.1 II 


1 64.6 j 




1 68 1 


1 ^6.3 II 


1 26.5 j| 


1 68.2J 



56 jj 21.4! 



17.9 i 6077 



87 -i 17.2 il 



39.1 i 43.7 



125! 



10.4 



53.6 



36.0 I 



109 49X| 



114 



24 jl 2^1 



2.8 477T| 



19.3!^ 63T 



12.5 il 5^ 
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Faculty Assessment Survey Results 


Fall 1997 1 


Part C: 


For each of the following skills, please indicate: a) whether the skill is 


an intended learning outcome ofyour classand b) how often you directly assessthem. 


(You may assess skills whether or not you assign grades based on it.) 



























Percentage Distribution 






1 Percent) 




1 (for those answering Yes) 


jSkill 


1 Number 1 


Yes! 

f 


1 Numberi 


1 o;l 


1-5 1 


6-10 1 


11-15|| 


16+ 




















1 1 . Reading 


1 ^1 


70.4 i 


1 158 


1 1:9.1 


29.7 1 


20.3 


21.5 ] 


26.6 1 





















|2. Writing 


( 


80.7 1 


1 188 


1 0;,1|[ 


37.2 1 


28.2 


13.3 j 


21.3| 




















13. Mathematics 


1 ^1 


36.5 1 


1 


1 0.0 i| 


43.6 f 


16.7 1 


14.1 ;| 


25.6 ( 




|4. Listening 


Jl.,_ 230 j 


81.71 


1 180 


1 il 


25.6 ;| 


13.3 i| 


17.2 ;| 


41.1 j 




















|5. Speaking 


1 227 i| 


63.9) 


1 138 


1 ‘■'*'1 


41.3 f 


21.0 || 


15.2 i| 


21.01 




|6. Creative Thinking 


1 ’1 


81.0) 


I 179 


1 2-2 ;| 


33.5 1 


25.7 :j 


14.0 1 


24.6 j 




















;|7. Problem Solving 


1 241 \ 


84.21 

j 


1 195 


1 ':°!1 


33.8 ’1 


21.0 1 


16.9 1| 


27.2 j 


i 

1 

1 


















>|8. Analysis and Synthesis 


1 ^^*1 


88.3 j 


1 196 


1 


34.2 :| 


29.6 1 


12.8 !| 


23.0 j 


|9. Reasoning 


1 ^‘1 


91.91 

i 


r ^ 


1 'Oil 


28.8 -| 


28.8 1 


17.1 :| 


24.4 j 




















|10. Aesthetic Responsiveness 


1 210 :| 




i . 21 


1 2-8 !| 


32.4 1 


29.6 1 


12.7 || 


22.5 i 




















l| 1 1 . Computer Literacy 


[■ ~224 f~ 


42.9 ) 


[ 90 


1 i 


46.7 1 


15.6 ,[ 


16.7 1 


20.0 1 


I 


















j[12. Social Interaction/Teamwork 


1 23T,j 


77.7 i 


1 174 


1 3'':l 


32.8 :| 


21.3 i| 


14.9 •[ 


27.6 J 


1 

i 


















|13. Value Judgements and 


1 


77.01 


1 169 


1 5,3 :| 


31.4 ,| 


26.6 ;| 




21.3 j 


Ijlndependent Decision Making 


















I 


















i|14. Transfer of Knowledge and 


1 ^'1 


88.8! 


1 197 j 


1 


30.5 ;| 


27.4 || 


14.7 1 


24.4 j 


i|Skills to a new Context 


















i 

1 


















|15. Effective Citizenship 


I 2TF:| 


56.0! 


1 ___ “ 


1 ® 0 '\ 


37.6 :| 


18.8 1 


12.8 1 


24.8 j 




j|16. Other 


1 


77.1 1 


1 10: 


1 *1 


40.0 1 


10.0 ;| 


20.0 ^1 


30.0J 



(Tod of Paee) 




Faculty Assessment Survey Results 



Fall 1997 



j|Part D: (Backside Questions) 



ijQuestion 



Percentage Distribution 



iNumber 



1 . I use assessment to determine learning! 
improvement 



248 



2. I or department do adequate diagnostic 
assessment 



246 



3^ I or department do adequate exit 
assessment in courses 






250 



4. I like to compare student performance 
semester to semester 



245 



5. Students are generally qualified in my: 
classes. 



244 



6. Palomar adequately assesses student 
skills prior to completion 



252 



7. Palomar ensures AA grads possess 
minimum skills & knowledge 



250 



9. I have enough resources to adequately; 
assess my students 



250 



4.0 



32.4 



10.4 



Assessment of Learning Project and Institutional Research and 
Planning - 12/97 



[d:\Assessmentof Learning Project\Faculty Assessment Survey 
Results.wb3] 



Strongly! 




No' 


1 Strongly] 


1 Agree! 


1 Agree 


|Opinion|Disagree| Disagree] 






9.3 


33.5 


26.6 


21.8 ; 


8.9 1 




9.3 


30.9 


16.7 ; 


30.1 1 


13.0 j 




18.4 j 

1 

\ 


55.4 


10.4! 


10.4) 


5.61 

1 




18.4 1 


32.2 


29.4 ; 


15.9 j 


4.1 




9.8 


55.3 


8.2, 


22.5 ! 


4.1 




2.0 j 


25.8 i 


42.9 1 


21.0! 


8.3 




8.4 


41.2 : 


34.4 j 


12.8 1 


3.2 1 

1 



8. Palomar should develop means for; 


249 1 


22.9 


51.0 


16.1 ! 


8.0] 


2.0] 


assessing AA candidates 








. i 







34.81 



18.4 



Appendix C -Interview Questions (Top of Page) 

Interview Questions 



1) Do you do any diagnostic entry assessment at the beginning of your 
courses? 

If yes, briefly describe this assessment and its goals. 

2) What assessment techniques do you employ to test understanding of your 
subject? 

How effective are they? 

3) How do you use assessments of student learning other than to assign grades? 

4) Do you have any formal or informal ways of finding out whether your 
students this semester are learning more or less than your students in 
previous semesters? 

If yes, describe these methods. 

5) I am going to give you a list of skills. 

How do you assess the acquisition of these skills? When? How often? 

In responding to the following questions, try to describe how you think 
Palomar College should be assessing students' skills and knowledge five 
years from now. Assume that resources, administrative support, and faculty 
support are available to develop new means of doing things. 

6) How should Palomar College assess the knowledge and skills of incoming 
students in order to place them in the proper programs and provide guidance 
toward a successful college career? 

7) Please refer to the list of skills in front of you. Feel free to 
comment on any or all of these skills in your response to the following 
questions: 

a) At what administrative level of the college can these skills best be 



assessed? The college, the division, the department, the classroom, or some 
other level? 

b) When and how often in the student's career should these skills be assessed? 

c) How should these skills be assessed? 

8) How should we find out whether our students this semester are learning 
more or less than students in previous semesters? 

9) How should we find out how well individual programs are performing? 

10) What specific skills from your discipline are essential for the 
educated layperson to acquire? 

List of Skills: 

Reading 
Speaking 
Writing 
Mathematics 
Listening 
Reasoning 
Problem Solving 
Creative Thinking 
Aesthetic Responsiveness 
Social Interaction and/or Teamwork 
Value Judgments and Independent Decision Making 
Transfer of Knowledge and Skills to a New Context 
Computer Literacy 
Effective Citizenship 
Analysis and Synthesis 
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Other (please specify) 

Appendix D - Interview Responses 
(Top ofPaee) 

1. Do you do any diagnostic entry assessment at the beginning of your courses? If 
yes, briefly describe this assessment and its goals. 

(A) Yes (12) 

- Diagnostic writing test 

- Reading inventory standardized tests: 

- Nelson-Denny (nationally normed inventory) 

- Test of Adult Basic Education Skills, reading section (for lower or developmental 
courses) 

-Quick assessment for each learner individually: San Diego Quick Assessment or wide 
range achievement test. 

- If new, they must take the ESL CELSA test and must place in level 4 

- Writing assessment 

- Dance technique very visible, observation 

- First project, solo project, assess tools; entering over several weeks 

- Need basic math skills to do well (13 question quiz) 

- Pre- and post-testing / survey test (oral and written language skills 

- Assessment and guidance (DSP&S) 

- Writing sample (support class for English 50) 

- Formal checklist of computer skills for computer classes in ESL 

-10 questions pre & post test used in geography & geology (NSF grant) 

(B) No(ll) 

- Informal observation 

- Pretty early; writing sample, brief research opinion profile 
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- Observe proficiency level in Spanish capability 

- Personal recommendation 

(C) Some / Informal / Interview (5) 

- To get into nursing program: Sci., Eng., very generic. Once in program, no pretest for 
specific courses, but we go over their folders from previous course(s?) 

- Students will introduce themselves and inform the class of what they want to learn and 
what they expect to learn from the course 

- Hand out prerequisite questions in lower level classes; help them to be successful and/or 
advise them to seek another class 

- Personal interviews, or they come in with all prerequisites fulfilled 



2. What assessment techniques do you employ to test understanding of 
your subject? How effective are they? 

- Homework/Assignments 4 

- Quizzes 1 1 

- Tests: 21 

- multiple choice 7 

- problem solving 1 

- analysis 1 

- true/false 3 

- fill-in 1 

- matching 1 

- oral 1 

- theory & practice 1 

- skills/hands-on 4 

- essay 5 read & follow directions 1 

- Project 5 

- Short Essay 1 
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- Essay 1 

- Short Paper 1 

- Research Paper 1 

- Book Report 2 

- Case Study 2 

- National Standardized Exam 1 

- Survey of Internship Supervisor 1 

- Interaction 1 

- Monthly Reports 1 

- Videotaping 1 

- Analyzing Scenes 1 

- Script Analysis 1 

- Presentation of Scene 1 

- Role Playing 1 

- Direct Debate 1 

- Writing Sample 1 

- Journals 1 

- Class Discussions 1 

- Worksheet Activities 1 
-Lab 2 

- Small Group 1 

- Group Assessments 1 

- Visual Evaluation (Dance) 1 

- Movement (Dance) 1 

- Hand-in Notes 2 

- Individual Assessments 1 




- Critique 1 

- Portfolios 1 

- Goal Paper 1 
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- IB test 1 



- Study and plan of care 1 

- Clinical assessment 1 

- State Board Exam 1 

- Conferencing 1 

- Internet use 1 

3. How do you use assessments of student learning other than to assign grades? 
Student Feedback: 

- Feedback to students 

- Informal obervation, feedback, one-to-one individual basis 

- Diagnostic, up-to-task feedback 

- Portfolios, relationships, constant feedback, individual instruction 

- Questions and answers in classroom discussions. Diagnostic feedback. 

Participation is very important 

- Objectives - employee feedback, student feedback. Job offers, promotion 

- To complete students' folders, then review their work on a weekly basis individually 
with them. 

Instructor Feedback: 

- Self-evaluation on my part 

- At the beginning, use for adjustment 

To guide what I need to focus on in the course 
Use to evaluate how I can change the course 

- Use a lot of self assessment curriculum and techniques - adjust way of working 

- Feedback for techniques as well as assessment 

- Feedback to find out his own effectiveness and to adjust emphasis when necessary 

ERIC 
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- Finding out what to teach and when to teach it 

- She begins teaching at the level where she finds the most errors 
Other: 

- to help students to learn. I see tests more effective as a learning device than to assign 
grades. 

- whether or not a student is progressing 

- check attendance: whether or not students' performance is related to their attendance 
-No 

- Students know what to expect when they go to the next course. 

- Make them help another student. "To teach is to learn" 

- Assess their progress, Feedback tool 

- I like to know where the class is at any time, do not like surprises 
Assess whether students are ready to go on to the next level 
Recommend to repeat sometimes 

- for them to understand responsibility for their own safety 

- Growth, creativity 

- Tap into life experience 
Communication skills, critiques 

- Student initiative, attitude in class, participate in discussion 

- Filling gaps in understanding teaching tool 

- Referral to counseling or other courses. Plan their future 

- To assess placement and progress through levels to determine hands-on work: safe 
practice 

4. Do you have any formal or informal ways of finding out whether your students this semester are learning more 
or less than your students in previous semesters? If yes, describe these methods. 

(A) Yes— Formah 



- Depends on the students (students more independent now) 

- Formal - testing and term paper 

- look at how well they are performing in their past exiting tests. Their affective score: 
how 

many positive responses they have. Compare the number of positive responses from this 
semester to the next. 

- Passing of English 50 final. Observe writing growth, track students 

- Begun a survey of students, their attitudes toward government and knowledge. 

- Intuition, looking at past work, use the same test but different versions, use own tests, 
reviewing assessment techniques 

- Keep accurate records of all the points received by my students on various assignments 
and I 

compare them semester to semester. I also keep some of the written work that my students 
have done and compare the quality of work. 

- Yes, I recycle exams. I have a file of exams and I make notes on them. I see negative 
trends 

- Formal: quizzes 

- High school students, compare final essays 

- He selects, eg. the "mineral" quiz to compare last semester with this semester. Also 
assessed 

with computer use vs. non-computer use. 

(B) Yes— Informal: 

- very informal, need to define some kind of post test. Thinking about mini-certificate for 
program 

- Can visually tell, by observation, can only go as fast as the group will go. Can tell with 
written 

work, and the way they respond in class discussions (Dance) 
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- Informal: Finishing class, test pieces on hardwood. Last year I had them restore a piece 
of 

furniture. Quality of the test pieces improved when they had to apply it immediately. 

- Informal: Quality of scene presentations 

- Observation and intuition. Observe patterns (waves) of students. 

- observation 

- Informal way: I interview students; I will have them explain their reasoning on certain 
topics 

(interactive courses, not in traditional courses) 

- Informal. One-on-one assessments - tracking students 
(C) No: 

- No, except our state and national board exams 
-No 

- No, maybe informal. Interview at beginning of semester. Fill out a sheet. 

-No 

- No formal ways 

- No formal ones - In the past, I gave similar final exam problems, but in most courses, no. 

- No, really - program evaluation 

- No, no experience in previous environment. Have interview at beginning of semester. 
-No, never done 

- No, the only formal assessment that serves their purpose is the Academic Placement to 
get into 

ESL 101. 

5. (Top ofPaee) 
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Skill i 

; 


When 


How 

often 


How to assess 


Reading 


ongoing 


Every 

class 


■i 

Apparent in scene presentation 


Writing 




j 


Minimal factor, keep a journal 


! Mathematics 






zero i 


1 Listening 


Ongoing i 


Every 

class 


Instructions, listening to partner 


Speaking 1 




Every 
class I 


Apparent in work, presentation 



Value Judgements and 
Independent Decision | 

Making 


Ongoing ■ 


Constantly 


Feedback sessions—peer evaluations, quality 
of choices 


Transfer of Knowledge 
and Skills to a new 
Context 






Confidence building 


Effective Citizenship 






Not assessed directly, but social responsibility 
is stressed 


Other (please specify) | 

j 









7. At what level should these abilities be assessed? 



Skill 


College 


Classroom | 


Dept/ 








Division 


Reading 


16 


7 


1/1 


Writing 


16 


8 


0/1 


Mathematics 


15 


7 I 


0/1 


Listening 


7 


11 i 


3/1 


Speaking 


7 


1^. _J 


2/1 


Creative Thinking 


2 


10 1 


2/1 


Problem Solving 


2 


11 


2/2 


Analysis and Synthesis 


2 


11 


2/2 * 


Reasoning 


2 


11 


2/2 


Aesthetic Responsiveness 


3 


10 


2/1 


Computer Literacy 


3 


10 


2/1 


Social Interaction and/or 


3 


10 


2/1 1 


Teamwork 






1 


-Value Judgements and 


4 


8 1 


3/1 1 


Independent Decision 
Making 






1 


Transfer of Knowledge and : 


4 


8 1 


2/1 


Skills to a new Context 




j 




Effective Citizenship 


3 


8 


2/1 


Other (please specify) 






0/1 


Study Skills 









How often should these skills be assessed? 



Continuously/Ongoing 


5 


Entry /Exit — Pre/Post 


5 


Whenever Students Ask for It 


1 


Exit 


1 



How should these skills be assessed? 

er|c 
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Method 


Number of 
Responses 


Written Instrument j 


2 


Interview j 


2 


Computerized Testing ' 


1 


Portfolios 


1 


"Light a fire under the faculty" 


1 



9. How should we find out how well individual programs are performing? 



Method 1 


Number of 
Responses 


Compare grades | 


1 


Grades no good 


1 


1 Retention 


1 


Self-assessment by depts. 


4 

i 


Faculty should evaluate faculty | 


1 


{ Exit assessment | 

! ; 


1 


Portfolios j 


1 


New assessment vehicle | 


1 




Standards j 


1 


I Goals and objectives in 5-year plan linked 
i to learning 


1 


i Key indicators of good progress 


1 


! Assessment center 


1 


j Track students 


1 


1 Compare pre-post test with other colleges | 


1 


i Standardized tests | 


1 


"I don’t like institutional review" j 

1 


‘ , i 



9. What specific skills from your discipline are essential for the educated 
layperson to acquire? 



Discipline 


Skills 1 


Geology 


Ability to make good decisions about where to I 

build houses; earthquake, flood, landslide safety | 


ESL 


Fluent English; ability to succeed in other classes j 


Nursing 


Understanding roles of health care workers; tools 
to make educated decisions about health care 


Performing Arts — Theater 


Presence, confidence, ability to speak at ease in 
public; empathy and compassion 


DSP&S 


Understand students with disabilities and what 
accommodations are appropriate 



Coop Ed. 


Bring theory into practice; how to make a living 


Political Science 


Knowledge for citizenship 


History 


Role as a citizen; reality based history 


Business 


Language of business; financial transactions 


DSP&S 


Communication and language 


Foreign Language 


Conversational skills; cultural awareness 


Art 


Awareness of what you see in media; aesthetic 
sense; paying attention with awareness to the 
world 


Mathematics 


Arithmetic, algebra, statistics 


Reading 


Read and listen at different levels; decode 
symbols 


English 


Reading, speaking writing, listening, problem 
solving, reasoning, computer literacy 


Drafting? 


Reasoning, geometric formulas, trig, 
systematized, well-organized thought 


? 


Ethics, problem solving, creative thinking, 
speaking, \vriting, ethical way of resolving 
problems 


Dental Assisting 


Effective, competent, safety 


English 


Reading, \vriting; ability to respond to literature 


Library Science 


Information competency; reading, speaking 
\vriting; Internet; finding information 


Life Sciences 


Knowledge of the human body and how it works 
in health and disease; separate hucksterism from 
legitimate medicine 



Graphic Communications 


Visual communication; need to be visually literate 
so can communicate with images as well as words 


Philosophy 


I 

Critical thinking, verbal skills, logical skills j 


Nursing 


Basic health care and what they can do to protect 
themselves through and from the health care 
system 


Art 


A taste of culture; basic drawing 


i ESL 


Reading and writing, spoken language; 
communication 


' Mathematics 


Arithmetic, algebra; how to interpret information, 
compare and contrast 


Cabinet Making and Furniture 


Problem solving; creative thinking; aesthetic 
development; independence 


Upholstery (ROP) 


How to be adept at using your hands 


i Performing Arts — Dance 


Physical, mind, feelings involves the whole 
person; release from stress; an outlet to maintain 
well being 



Appendix E - Budget 
(Top of Pa2e) 

Members of the Assessment of Learning Project Team: 

Teresa Laughlin 
John Tagg 
Fari TowFiq 
Cynthia Watson 

Instructional Planning Committee Subcommittee on Assessment: 

Aura Cuevas 

Teresa Laughlin 

Michael Rourke 

Pat Schwerdtfeger 

John Tagg 

Lise Telson 
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Dr. Robert Barr and the Office of Institutional Research and Planning have provided 
invaluable help and support in the work of the Project, 
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